Diary 2024-12-31
o1 Pro.icon
Tasks such as automating testing and writing Dockerfiles are generally considered part of "DevOps" or "Infrastructure as Code, IaC". The following designations are also used, depending on the specific purpose and content of the project. Automating test cases and setting up Dockerfiles are part of the process of building a CI/CD pipeline for software development.
Infrastructure as Code (IaC)
The act of writing a Dockerfile is part of "IaC," managing infrastructure as code.
Like Terraform and Ansible, it corresponds to a method of writing and managing environments in code and automating them. Automation Engineering
The term encompasses activities that automate the process of testing and infrastructure building.
This may include test automation and managing infrastructure configurations, especially for operational reliability.
nishio.icon
They can be interpreted as the process of rewriting documents that were once written for human reading in natural language to be read by machines in code. as of 2024, the development of LLM has made it possible for AI to read natural language. How will this change in the future?
o1 Pro.icon
In an age when text is understood by machines as it is, the following changes can be expected
Changing role of codes
Until now, formal coding has been mandatory for machines to read.
In the future, code is likely to become "less visible back-end after completion," with code being generated directly from natural language-based specifications and requirements.
Focus on natural language interactions
The development of LLM will seamlessly connect requirements definition and documentation writing with code generation and configuration file updates.
There will be a growing trend for the business side to communicate requirements in natural language and for the engineering part to be converted and optimized by LLM.
Emphasis on verification and coordination
Engineers focus on "verification and adjustment" to ensure that the generated code and settings are correct and how to operate and manage them.
Even with increased automation, human evaluation and judgment are more important because of unexpected behavior and the need to ensure safety.
Shift from "code-centric" to "dialogue-centric
Until now, the code has been the main focus, with documentation as an adjunct.
Possibility to move to a form where documents and requirements are generated and updated through "interaction with LLM" and code is output by LLM as needed.
In short, AI will be able to automatically generate and modify technical tasks from natural language interactions, and engineers will increasingly focus on "areas requiring judgment" such as verification, operation, and optimization.
nishio.icon
Is it appropriate for rigorous verification to be done by human minds? Will the weight of formal verification methods increase?
o1 Pro.icon
There is a limit to how rigorously humans can visually verify specifications and code at the natural language level.
The more LLMs automatically generate code and settings, the more important the following formal verification becomes as a method to ensure "correctness.
Formal specifications and proofs
Use frameworks such as TLA+ or Coq to mathematically prove system behavior. Verification accuracy can be increased by reducing abstract natural language-based specifications into formalized specification descriptions.
Advanced static analysis tools
Static analysis tools combined with AI will increase the possibility of automatically detecting code errors and security holes.
This is effective in improving final quality because it allows checking of points that people tend to miss.
Humans focus on abstraction and judgment
It is more efficient to focus on final decisions, abstraction, and operational verification in combination with the above tools, rather than having humans review every line of code in their heads.
Combining formal methods with human review facilitates both safety and availability.
In general, the utilization of formal verification is expected to increase more and more due to the limitations of rigorous verification by humans using only their minds.
---
This page is auto-translated from /nishio/日記2024-12-31 using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. I'm very happy to spread my thought to non-Japanese readers.